RidgeRun Metadata/Streaming Protocols/UDP

Follow Us On

RidgeRun Metadata

Table of Contents [Sticky]

Introduction

User Datagram Protocol (UDP) is a transport-layer protocol in the IP suite. It is connectionless and unreliable, meaning it sends data without session setup and with no guarantees of delivery, order, or error correction.

Its lightweight design and low overhead make it ideal for latency-sensitive applications like real-time streaming, VoIP, online gaming, and DNS.

Key Characteristics

No session setup (connectionless).
No delivery or order guarantees (unreliable).
Low overhead (small header).
Optimized for speed and low latency.
Port-based process communication.

UDP delivers data as datagrams, including source/destination ports and an optional checksum for integrity.

Benefits and Limitations

Benefits

UDP has several advantages that make it ideal for real-time and latency-sensitive applications:

Low latency: No connection setup or acknowledgments, making it faster than TCP.
Low overhead: Simple 8-byte header reduces processing and bandwidth usage.
Efficient: Minimal resource usage enables lightweight communication.
Scalable: Handles many simultaneous streams, useful in conferencing and live streaming.
Real-time friendly: Prioritizes speed over reliability for time-sensitive data.

Limitations

However, UDP also has key limitations that make it unsuitable for reliable transmission:

No delivery guarantee: Lost packets are not retransmitted.
No ordering: Datagrams may arrive out of sequence.
No error correction: No mechanisms to detect or fix lost/corrupted data.
No congestion control: Can worsen packet loss under heavy traffic.
Security risks: Lacks encryption and authentication.

RidgeRun compatible products

Among the RidgeRun products that can be used with the UDP protocol are:

SEI: Embeds metadata in H.264/H.265 SEI NAL units, directly compatible with UDP.
OBU: Inserts metadata into AV1 streams using Open Bistream Units, suitable for UDP.
MPEG-TS: Supports metadata injection/extraction; TS is natively UDP-friendly.
RTMP (FLV over UDP): Though RTMP uses TCP, flvmux can generate FLV streams for raw UDP use in controlled setups.

Usage Implications (UDP + Metadata)

Method	Description	Implications
UDP + SEI	SEI metadata is embedded in H.264/H.265 video streams using SEI NAL units. This method is container-agnostic and works seamlessly over RTP/UDP.	Benefits Precise synchronization: Metadata is tied to video frames. Simple parsing: No external demuxing; metadata is part of the stream. Limitations Fragmentation risk: Large SEI messages (>1 KB) may exceed MTU. Timing sensitivity: Packet loss and jitter can affect frame/metadata alignment; buffering may be needed. Limited compatibility: Most commercial video players and decoders ignore SEI metadata. Extracting this information requires specialized tools, such as `seiextract`.
UDP + OBU (AV1)	OBU metadata is embedded directly in the AV1 video bitstream. Compatible with any container or protocol that supports AV1, including UDP.	Benefits Frame-level sync: Metadata OBUs can be tied to individual frames. No demuxing required: Since metadata is part of the video bitstream, it can be accessed directly from the stream using tools like `obuextract`. Limitations Fragmentation: OBUs > ~1200 bytes may be split; partial loss affects metadata. Timing sensitivity: Reordering and jitter may require buffering to maintain sync. Limited support: Most commercial players don’t process metadata OBUs. You need dedicated tools, such as `obuextract`, to extract and interpret this data.
UDP + MPEG-TS	MPEG-TS is a widely used container format in IPTV and broadcast environments, commonly transported over UDP. It is designed for efficient multiplexing of audio, video, and metadata streams in a packetized format, making it well-suited for real-time transmission.	Benefits Synchronous or asynchronous transport: Metadata can be sent in sync with video frames or independently, depending on the use case and stream design. Native support: Some commercial players can extract standardized metadata (KLV) from MPEG-TS streams without requiring custom software or affecting video/audio decoding. Limitations Fragmentation risk: Loss of any fragment drops the full 1316-byte group. Buffering tradeoff: Less buffering = glitches; more = higher latency. Multicast sensitivity: Depends on network setup. Loose frame association: Since metadata is transmitted in separate streams (e.g., private PES), it is not inherently tied to individual video frames. Synchronization must be handled via timestamps, which may introduce complexity.
UDP + FLV (RTMP)	This setup sends FLV streams over UDP instead of TCP. It is not true RTMP protocol—no handshakes or control mechanisms are used.	Benefits In-band metadata: No side channel—metadata travels inside the FLV container. Limitations Not RTMP-compliant: No handshake or RTMP protocol features. Limited compatibility: Doesn't work with standard RTMP servers (e.g., YouTube, Wowza). No session control: Lacks acknowledgments, ping/pong, and control messages. Non-standard metadata: Custom metadata injection into FLV is not part of the official spec. Most players ignore these tags, so specialized extraction tools (such as RidgeRun’s parser) are required.

Examples

This section presents reference pipelines that demonstrate various combinations of metadata transmission and reception using RidgeRun’s products over the UDP protocol. Each example typically includes a sender, a receiver, and the corresponding output, showcasing the complete end-to-end flow. The pipelines highlight different metadata injection and extraction methods, such as SEI, OBU, MPEG-TS and FLV (RTMP).

SEI

Sender

gst-launch-1.0 videotestsrc is-live=true num-buffers=5 ! x264enc ! seiinject metadata="HELLO WORLD" ! rtph264pay ! "application/x-rtp,media=video,clock-rate=90000,encoding-name=H264" ! udpsink host=127.0.0.1 port=5555

Receiver

GST_DEBUG=*seiextract*:MEMDUMP gst-launch-1.0 udpsrc port=5555 ! "application/x-rtp,media=video,clock-rate=90000,encoding-name=H264" ! rtph264depay ! h264parse ! seiextract ! fakesink

Output

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:05.950631067 24220 0x5752f2a71520 DEBUG             seiextract gstseiextract.c:238:gst_sei_extract_set_caps:<seiextract0> set_caps
0:00:05.950686432 24220 0x5752f2a71520 DEBUG             seiextract gstseiextract.c:244:gst_sei_extract_set_caps: parsing caps: video/x-h264, stream-format=(string)avc, alignment=(string)au, codec_data=(buffer)0142c015ffe1001c6742c015d90141fb016a0c0c0d4a000003000200000300791e2c5c9001000468cb8cb2, level=(string)2.1, profile=(string)constrained-baseline, pixel-aspect-ratio=(fraction)1/1, width=(int)320, height=(int)240, framerate=(fraction)30/1, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, colorimetry=(string)bt601, parsed=(boolean)true
0:00:05.950926736 24220 0x5752f2a71520 LOG               seiextract gstseiextract.c:363:gst_sei_extract_prepare_output_buffer:<seiextract0> prepare_output_buffer
0:00:05.950945898 24220 0x5752f2a71520 LOG               seiextract gstseiextract.c:291:gst_sei_extract_extract_h264_data:<seiextract0> Not the expected payload type: 5

0:00:05.950957409 24220 0x5752f2a71520 MEMDUMP           seiextract gstseiextract.c:299:gst_sei_extract_extract_h264_data:<seiextract0> ---------------------------------------------------------------------------
0:00:05.950962593 24220 0x5752f2a71520 MEMDUMP           seiextract gstseiextract.c:299:gst_sei_extract_extract_h264_data:<seiextract0> The extracted data is: 
0:00:05.950968256 24220 0x5752f2a71520 MEMDUMP           seiextract gstseiextract.c:299:gst_sei_extract_extract_h264_data:<seiextract0> 00000000: 48 45 4c 4c 4f 20 57 4f 52 4c 44 00              HELLO WORLD.

OBU

Sender

gst-launch-1.0 videotestsrc is-live=true num-buffers=15 ! "video/x-raw, width=320, heigth=240" ! av1enc cpu-used=8 ! obuinject metadata="Hello World" ! qtmux ! udpsink host=127.0.0.1 port=5000 -v

Receiver

GST_DEBUG=*obu*:MEMDUMP gst-launch-1.0 udpsrc port=5000 ! qtdemux ! obuextract ! fakesink silent=false -v

Output

/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event   ******* (fakesink0:sink) E (type: tag (20510), GstTagList-stream, taglist=(taglist)"taglist\,\ video-codec\=\(string\)AV1\;";) 0x7f31e0003720
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event   ******* (fakesink0:sink) E (type: tag (20510), GstTagList-global, taglist=(taglist)"taglist\,\ datetime\=\(datetime\)2024-02-12T23:10:14Z\,\ container-format\=\(string\)Quicktime\;";) 0x7f31e0005420
0:00:05.563025033 220219 0x56463b9d75e0 MEMDUMP           obuextract gstobuextract.c:286:gst_obu_extract_prepare_output_buffer:<obuextract0> ---------------------------------------------------------------------------
0:00:05.563033177 220219 0x56463b9d75e0 MEMDUMP           obuextract gstobuextract.c:286:gst_obu_extract_prepare_output_buffer:<obuextract0> The extracted data is
0:00:05.563042096 220219 0x56463b9d75e0 MEMDUMP           obuextract gstobuextract.c:286:gst_obu_extract_prepare_output_buffer:<obuextract0> 00000000: 48 65 6c 6c 6f 20 57 6f 72 6c 64                 Hello World     
0:00:05.563045653 220219 0x56463b9d75e0 MEMDUMP           obuextract gstobuextract.c:286:gst_obu_extract_prepare_output_buffer:<obuextract0>

MPEG-TS

Sender

gst-launch-1.0 -e metasrc period=1 metadata=0x01020304 ! 'meta/x-klv' ! mpegtsmux name=mux ! udpsink port=3001 \
videotestsrc is-live=true ! 'video/x-raw,format=(string)I420,width=320,height=240,framerate=(fraction)30/1' ! x265enc ! mux.

Receiver

gst-launch-1.0 -e udpsrc port=3001 ! 'video/mpegts, systemstream=(boolean)true, packetsize=(int)188' ! tsdemux name=demux demux. ! queue !  \
h265parse ! 'video/x-h265, stream-format=byte-stream, alignment=au' ! avdec_h265 ! autovideosink sync=false demux. ! queue ! 'meta/x-klv' ! metasink async=false

Output

metasink 0x562d51fd39e0: received KLV metadata buffer (size=4)
  Hex Dump: 01 02 03 04
  Timestamp: 0:00:02.000000000

FLV

Sender

gst-launch-1.0 -v videotestsrc is-live=true pattern=ball   ! x264enc bitrate=500 speed-preset=ultrafast key-int-max=30   ! flvmux name=mux meta-string="Hello from FLV"   ! udpsink host=127.0.0.1 port=5000

Receiver

GST_DEBUG="flv*:6,gsttag:6,flvdemux:6" gst-launch-1.0 -m -v   udpsrc port=5000 caps="video/x-flv"   ! flvdemux name=d   ! fakesink silent=false

Output

39:gst_flv_demux_parse_tag_script:<d> we have a metadata script object
0:00:08.870420459 25888 0x5bd2744fad20 DEBUG               flvdemux gstflvdemux.c:589:gst_flv_demux_clear_tags:<d> clearing taglist
0:00:08.870429362 25888 0x5bd2744fad20 DEBUG               flvdemux gstflvdemux.c:661:gst_flv_demux_parse_tag_script:<d> there are approx. 1 elements in the array
0:00:08.870434478 25888 0x5bd2744fad20 DEBUG               flvdemux gstflvdemux.c:350:gst_flv_demux_parse_metadata_item:<d> tag name meta-string, tag type 2
0:00:08.870438960 25888 0x5bd2744fad20 DEBUG               flvdemux gstflvdemux.c:419:gst_flv_demux_parse_metadata_item:<d> meta-string => (string) Hello from FLV

References

❯

Share This Page